חלק מהשקפים מעובדים משקפים של פרדו דוראנד, טומס פנקהאוסר ודניאל כהן-אור קורס גרפיקה ממוחשבת 2009/2010 סמסטר א' Image Processing

Transcription

1 חלק מהשקפים מעובדים משקפים של פרדו דוראנד, טומס פנקהאוסר ודניאל כהן-אור קורס גרפיקה ממוחשבת 2009/2010 סמסטר א' Image Processing 1

2 What is an image? An image is a discrete array of samples representing a continuous 2D function Continuous function Discrete samples 2

3 Amplitude Converting to digital form Convert continuous sensed data into digital form Quantization Sampling 3

4 Sampling and Reconstruction Sampling Reconstruction

5 Sampling and Reconstruction Figure 19.9 FvDFH

6 Sampling Theory How many samples are required to represent a given signal without loss of information? What signals can be reconstructed without loss for a given sampling rate?

7 How many samples needed to reconstruct this image? Sampling And this one? And this one? 7

8 What happens when we use too few samples? Aliasing Aliasing Figure FvDFH

9 Spectral Analysis So our image (function f(x,y)) describes how the signal changes over time (x and y axes) Aliasing occurs when we use too few samples (what is enough?) The more an image changes, the more we need to sample it. How do we measure how fast a signal changes? 9

10 Spectral Analysis Spatial domain: Function: f(x) Filtering: convolution Frequency domain: Function: F(u) Filtering: multiplication Any signal can be written as a sum of periodic functions.

11 Fourier Joseph Fourier discovered in 1822 that Any periodic function can be expressed as the sum of sines and/or cosines if different frequencies (Fourier Series) Even functions that are not periodic can be expressed as the integral of sines and/or cosines (Fourier Transform) Initial application was in heat diffusion 11

12 Fourier Transform (1D) Figure 2.6 Wolberg

13 Fourier transform: Fourier Transform (1D) j2 t F( ) f ( t) e dt Inverse Fourier transform: ( ) ( ) j 2 t f t F e d The frequency variable μ depends on the units of t

14 The Fourier Transform of Sampled Functions In our case (images) we do not have a continuous function We won t go into detail, but it can be shown that 1 n F( ) F( ) T T n Sampling Can t capture all frequencies (I.e. all details) Therefore f(t) will be called Band-limited 14

15 The Fourier Transform of Sampled Functions We can recover f(t) from its sampled version if we can isolate F(μ) The Sampling Theorem: No information is lost if the sample rate is greater than twice the highest frequency Also called the Nyquist rate 15

16 Extending to 2D Let f(t,z) be a function of two variables, then j2 ( t z) F(, ) f ( t, z) e j2 ( t z) f ( t, z) F(, ) e d d Like before we can extend this to discrete functions and images 16

17 Some images and their transforms 17

18 Image Processing Pixel operations Add random noise Add luminance Add contrast Add saturation Filtering Blur Detect edges Sharpen Emboss Median Quantization Uniform Quantization Floyd-Steinberg dither Warping Scale Rotate Warps Combining Composite Morph

19 Adjusting Brightness Simply scale pixel components Must clamp to range (e.g., 0 to 1) Original Brighter

20 Adjusting Contrast Compute mean luminance L for all pixels luminance = 0.30*r *g *b Scale deviation from L for each pixel component Must clamp to range (e.g., 0 to 1) L Original More Contrast

21 Image Processing Pixel operations Add random noise Add luminance Add contrast Add saturation Filtering Blur Detect edges Sharpen Emboss Median Quantization Uniform Quantization Floyd-Steinberg dither Warping Scale Rotate Warps Combining Composite Morph

22 Linear Filtering (Spatial Domain) Convolution Each output pixel is a linear combination of input pixels in neighborhood with weights prescribed by a filter Filter = 22

23 Adjust Blurriness Convolve with a filter whose entries sum to one Each pixel becomes a weighted average of its neighbors Original Blur What do you think happens in the frequency domain? Filter =

24 More on blur (lowpass filters) We can either take a uniform kernel (mean filter) Or a Gaussian kernel A Gaussian kernel tends to provide gentler smoothing and preserve edges better

25 Edge Detection Convolve with a filter that finds differences between neighbor pixels Original Detect edges 1 Filter =

26 Sharpen Sum detected edges with original image Original Sharpened Filter =

27 Emboss Convolve with a filter that highlights gradients in particular directions Original Embossed Filter =

28 Non-linear filtering Any operation on a neighborhood around each pixel For example: Selecting the median value of the neighborhood Original 3x3 5x5 7x7 11x11 15x15 28

29 Image Processing Pixel operations Add random noise Add luminance Add contrast Add saturation Filtering Blur Detect edges Sharpen Emboss Median Quantization Uniform Quantization Floyd-Steinberg dither Warping Scale Rotate Warps Combining Composite Morph

30 Image Warping Move pixels of image Warp Source image Destination image

31 Image Warping Issues How do we specify where every pixel goes? (mapping) How do we compute colors at destination pixels? (resampling) Warp Source image Destination image

32 Example Image Scaling (x,y ) = (sx*x, sy*y); I(x,y ) =? 32

33 Image Warping Image warping requires resampling of image Resampling 33

34 BACK TO SAMPLING 34

35 Aliasing (again) In general: Artifacts due to under-sampling or poor reconstruction Specifically, in graphics: Spatial aliasing Temporal aliasing Under-sampling Figure FvDFH

36 Spatial Aliasing Artifacts due to limited spatial resolution

37 Spatial Aliasing Artifacts due to limited spatial resolution Jaggies

38 Temporal Aliasing Artifacts due to limited temporal resolution Strobing Flickering

39 Temporal Aliasing Artifacts due to limited temporal resolution Strobing Flickering

40 Temporal Aliasing Artifacts due to limited temporal resolution Strobing Flickering

41 Temporal Aliasing Artifacts due to limited temporal resolution Strobing Flickering

42 Sample at higher rate Not always possible Doesn t always solve problem Antialiasing Pre-filter to form bandlimited signal Form bandlimited function (low-pass filter) Trades aliasing for blurring

43 Image Processing Real world Sample Discrete samples (pixels) Reconstruct Reconstructed function Transform Transformed function Filter Bandlimited function Sample Discrete samples (pixels) Reconstruct Display

44 Image Processing Real world Sample Discrete samples (pixels) Reconstruct Reconstructed function Transform Transformed function Filter Bandlimited function Sample Discrete samples (pixels) Reconstruct Display Continuous Function

45 Image Processing Real world Sample Discrete samples (pixels) Reconstruct Reconstructed function Transform Transformed function Filter Bandlimited function Sample Discrete samples (pixels) Reconstruct Display Discrete Samples

46 Image Processing Real world Sample Discrete samples (pixels) Reconstruct Reconstructed function Transform Transformed function Filter Bandlimited function Sample Discrete samples (pixels) Reconstruct Display Reconstructed Function

47 Image Processing Real world Sample Discrete samples (pixels) Reconstruct Reconstructed function Transform Transformed function Filter Bandlimited function Sample Discrete samples (pixels) Reconstruct Display Transformed Function

48 Image Processing Real world Sample Discrete samples (pixels) Reconstruct Reconstructed function Transform Transformed function Filter Bandlimited function Sample Discrete samples (pixels) Reconstruct Display Bandlimited Function

49 Image Processing Real world Sample Discrete samples (pixels) Reconstruct Reconstructed function Transform Transformed function Filter Bandlimited function Sample Discrete samples (pixels) Reconstruct Display Discrete samples

50 Image Processing Real world Sample Discrete samples (pixels) Reconstruct Reconstructed function Transform Transformed function Filter Bandlimited function Sample Discrete samples (pixels) Reconstruct Display Display

51 Frequency domain Ideal Bandlimiting Filter Spatial domain Sinc( x) sin x x Figure 4.5 Wolberg

52 Convolution Practical Image Processing Finite low-pass filters Point sampling (bad) Triangle filter Gaussian filter Real world Sample Discrete samples (pixels) Reconstruct Reconstructed function Transform Transformed function Filter Bandlimited function Sample Discrete samples (pixels) Reconstruct Display

53 Convolution with triangle filter Triangle Filter Input Output Figure 2.4 Wolberg

54 Gaussian Filter Convolution with Gaussian filter Input Output Figure 2.4 Wolberg

55 AND BACK TO WARPING 55

56 Image Resampling What if we are resampling a 2D image? (u,v)

57 Image Resampling Compute weighted sum of pixel neighborhood Output is weighted average Kernel Function (u,v) W dst(u,v)=0; for(ix=u-w;ix<=u+w;ix++) for(iy=v-w;iy<=v+w;iy++) d=dist between (ix,iy) and (u,v) dst(u,v) += k(ix,iy) * src(ix,iy) d (ix,iy)

58 Image Resampling For isotropic Triangle and Gaussian filters, k(ix,iy) is a function of d and w (u,v) W d (ix,iy)

59 Triangle Filtering (width <= 1) Bilinearly interpolate four closest pixels a = linear interpolation of src(u 1,v 2 ) and src(u 2,v 2 ) b = linear interpolation of src(u 1,v 1 ) and src(u 2,v 1 ) dst(x,y) = linear interpolation of a and b (u 1,v 2 ) a (u 2,v 2 ) (u,v) (u 1,v 1 ) b (u 2,v 1 )

60 Kernel is a Guassian function Gaussian Filtering (u,v) w 3 d (ix,iy)

61 Image Scale Scale (src, dst, sx, sy): w max(1/sx,1/sy); for (int ix = 0; ix < xmax; ix++) { for (int iy = 0; iy < ymax; iy++) { float u = ix / sx; float v = iy / sy; dst(ix,iy) = resample(src,u,v,k,w); } } v (u,v) y (x,y) u Scale 0.5 x

62 How do we resample? Point sampling (N.N) Simple but causes aliasing Triangle and Gaussian Algorithm as we saw earlier Float resample(src,u,v,w) { int iu = round(u); int iv = round(v); return src(iu,iv); } 62

63 Original דוגמא 50% Nearest Neighbor 50% Bilinear 50% Nearest Neighbor 50% Bilinear 63

64 Original דוגמא 50% Nearest Neighbor 50% Bilinear 50% Nearest Neighbor 50% Bilinear 64

65 Original דוגמא 50% Nearest Neighbor 50% Bilinear 50% Nearest Neighbor 50% Bilinear 65

66 Image Warping (in General) Reverse Mapping 66

67 Image Warping (in General) Alternative (forward) 67

68 Project #1 Image Processing Scaling up and down 68

69 שאלת התרגיל מה יותר קשה, להגדיל או להקטין תמונה? 69

70 THE END 70